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ABSTRACT 

A new system for assessing student ratings of 
instruction has been implemented at the University of Washington. Two 
major changes are in evidence. Firsts there is an explicit 
recognition that student ratings can and do serve multiple functions^ 
and the same evaluative questions are not necessarily appropriate for 
each. Secondly, there is an explicit recognition that adequate 
diagnostic information cannot be efficiently provided instructors 
with use of a common set of evaluative questions for all classes. 
Student Rating Forms A through E are appended. (Author/BJG) 
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A Brief Description of the Student lutings Forms 
of the University of Washington 
Instructional Assessment System 
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A new system for assessing student ratings of instruction has 
been implemented ac the University of V/ashington. Tv;o major changes 
are in evidence. First, there is an explicit recognition that 
student ratings can and do serve multiple functions, and the same 
evaluative questions are not necessarily appropriate for each. 
Secondly, there is an explicit recognition that adequate diagnostic 
information cannot be efficiently provided instructors with use of 
a common set of evaluative questions for all classes. 
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A Brief Description of the Student Ratings Forms of the University 
of Washington Instructional Assessmeij.t System 

G. M. Gillmore 

Introduction 

A new system for assessing student ratings of instruction has been 
implemented Fall Quarter, 197^, at the Lfaiiversity of Washington. The 
basic data collection device is optically scannable sheets, as in the 
past; however, two major changes are in evidence. First, there is an 
explicit recognition that student ratings can and do serve multiple 
functions, and the setme evaluative questions are not necessarily a.ppro- 
priate for e&ch. Secondly, there is sm explicit recognition that adequate 
diagnostic inforaiation cajinot be efficiently provided instructors with 
use of a common set of evaluative questions for all classes. 

Multiple functions . If we ignore research, an institution of higher 
learning consists basically of individuals within three groups, adminis- 
trators, instructors, and students. Each of these groups can make use of 
student ratings of instruction for it b own purposes. Administrators need 
largely comparative information for ranlc, pay and tenure decisions, as 
well as to make decisions about course offerings and teaching assigniuents- 
Instructors need information to improve their courses and instructional 
methods. Students need information to help them choose courses effec- 
tively. These three functions could be termed normative, diagnostic, and 
informative, respectively. 

The content of items for each of these functions needs to be 
somewhat different. For the normative function, items should be very 
global in nature, implying no basic philosophy of _instziictJ.Qn, and allow- 
ing little or no possibility of being invalid for a specific class. 
Otherwise, comparisons among a variety of classes will not be reasonable. 
For the diagnostic function, items need to relate to what is actually 
happening within a class, and the results of polling students should 
potentially have direct implications for improving the course or instruc- 
tion, or at least in the identification of areas for which one may need 
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to collect more specific diagnostic information. Finally, informative 
items should be directed tov;ard those specific aspects of a course or 
instructor which provide useful information to students \dien choosing 
specific courses or programs. 

Multiple forms . In almost any instructional setting with more than 
one instructor, and more than one course, a variety of instructional 
goals and techniques are readily visible. At a college or university, 
this is undeniably the case. One set of evaluative questions fpr diag- 
nostic purposes cannot at once satisfy all types; e.g., a large lecture 
format and a small seminar format. Several solutions to this problem 
are availalDle. The extreme solution is to have each instructor choose 
his own items. This has several shortcomings. First, instructors need 
to make a serious time commitment to the item selection process. Pjany 
are unwilling to allocate their time in this way. Secondly, many in- 
structors lack the expertise to be effective item writers or even 
choosers. Thirdly, when administering a large program, distinct items 
for each class present some serious logistic problems. Finally, com- 
parison information is either impossible or extremely expensive to 
provide. Thus, that solution was rejected at our current level of 
techriologyv"^ 

Another solution, more common, is for each discipline to have its 
own form. The potentially large number of distinct disciplines makes 
this solution unattractive but, more important, the great variety of 
instructional goals and techniques found ^within most departments makes 
this approach hardly a solution at all. The problem of appropriate items 
seems less a discipline-oriented problem and more an instructional tech- 
nique problem. 

Our approach was to isolate broad course types which cut across 
discipline lines. An extensive analysis of actual courses yielded five 
such types, aj^though future analysis may reveal others, and assuredly 
some classes may not fit well into any category. 

The Forms 

In our work in developing new instruments for assessing student 
attitudes toward instruction, we tried to design forms which would make 
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differentiations among the three functions in terms of l) directions 
given students; 2) the content of the items ; and 5) vrtiat ie done with the 
results. Furthermore; we tried to satisfy the diagnostic function more 
completely by creating separate fonas; each tailored to a broad course 
type. 

Succeeding general instructions and demographic information; the 
forms contain basically five sections; three primarily for diagnostic 
feedback for the instructor; one primar'.ly for administrative evaluation^ 
and one primarily for student information. Each section is preceded by 
brief but distinct directions to students indicating the purpose of the 
itemfl. The five forms are found in Appendix A, I shall discuss each 
section in turn. 

General items . The items contained in section 1 are designed for 
the normative function. Notice that the section is brief; having four 
itemS; and each iter is very global in nature. The purpose of these items 
is to gain a very general assessment of students' attitudes toward the 
course as a whole; the content of the course; and two important components 
of instruction. These items appear on all forms. Their global nature 
gives confidence that none are invalid for any class. AlsO; their inclu- 
sion allows comparisons to be made university-wide; college-wide;' 
department -wide; etc. They also allow other comparisons which may have 
some importance for program evaluation; e.g.; one department versus 
another. 

We have chosen the following six response categories for use for 
these and all other items: Excellent; Very Good^ Good^ Fair^ Poor, Very 
Poor. These categories were chosen rather than the more common Likert 
categories; Strongly Agree to Strongly Disagree (at the expense of 
Sreater difficulty in writing coherent items); for two reasons. First, 
actual responses to these categories; and class m^ans, are more readily 
interpretable. It is easier to \anderstand that on the average you are 
"good" than you are "agree." Secondly; student ratings have a tendency 
to bunch up at the favorable end. There is evidence that use of both an 
Excellent and a Very Good category at the favorable end yield more between 
class variance, i.e.; discrimination, than a four or five point Likert 
scale. 



Diagnostic items . Diagnostic-type items tailored for five basic 
course types are found in section 2 of each form. These it^s are meeuit 
to help instnxctors discover weaknesses in their courses and teaching. 
However; at this level; items are still common within course types ; thus 
allowing normative comparisons with other similar courses. Specific 
items were determined both by content analyses of courses and by inter- 
views with more than one hundred randomly chosen instructors. 

Form A was designed primarily for small lecture-discussion type 
courses. Items primarily emphasize the clarity and quality jf informa- 
tion transmitted; but items dealing with interaction between instructor 
and student are included as well. - - 

Form B was designed prima.rily for large lecture-type classes; where 
little or no in- class interaction between instructor and student is 
practiced. ThuS; items strongly emphasize course organization and infor- 
mation trajismitted. 

Form C was designed for seminar discussion-type classes which 
include a minimal amount of formal lecturing by the instructor. The items 
emphasize quality of discussion as well as course organization and 
interest level. 

The items for Form D are tailored to those classes whose purpose is 
the teaching of problem-solving or heuristic methods. Clear expleuiationS; 
dealing with student difficulties and quality of problems are emphasized. 
\ There are classes in a number of disciplines which are skill 
oriented and in which students get ''hands on" experiences related to 
fUwure occupational demands; e.g.; nursing clinical; art studiO; 
social-work field experience; etc. Form E is designed for these classes; 
with items dealing with the special considerations of this type of course. 

Student items . Items to provide student information are found in 
section 3 of each form. By 'piggy- *':ing" this section onto the foiTii; 
information for students can be provided rather cheaply. (We still adopt 
the position that each instructor must approve; in adveuace; of having 
these items published for him or her» ) Furthermore; it allows selection 
of items which speak to student concerns. These items are also common to 
all forms ; thus implicitly implying that student concerns do not differ 
significantly over courses. 
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instrjitor. 
co-,rse: 



EDUCATIONAL ASSESSMENT CENTtR 
UNIVERSITY OF WASHINGTON 



SECTION. 



DlR-CTlON=^- VOU MAY REIU^N THIS QUESTIONNAIRE COMPLETELY OR PARTIALLY UNANSWERED WITHOUT PENALTY. 
USE A NO. 2 PENCIL AND MAKE ^MRKS FIRM BUT NOT SHINY. DO NOT CROSS OUT GRIDS, 
DO NOT WAKE STRAY MARKS. ERASE CLEANLV IF YOU CHANGE AN ANSWER. 



AH^N REGISTERING. WAS THIS A 


COURSE YOU WANTED TO TAKE? YES 


NO NEUTRAL 


STUDENT RATING 


' ^ THIS -IIJRSE. 

b H.S .OURSEi MAJOR 


IN YOUR MINOR OR A DISTRIBUTION 
PROGRAM REQUIREMENT REQUIREMENT 


AN 

ELECTIVE 


OTHER 


FORM 


...-S:>. FRESHVAN 


SOPMOMORE JUNIOR SENIOR 


GRADUATE 


OTHER 


B 


1 GRADE YDU EXPECT TO RECEIVE: 

! 


A B C D E PASS 







IVPORTAMT- IN RATING THIS COURSE, RESPOND TO EACH ITEM 

CAREFULLY AND THOUGHTFULLY. AVOID LETTING YOUR 
RESPONSES TO SOME ITEMS INFLUENCE YOUR RESPONSES 
TO OTHERS. KEEP THE PURPOSE OF EACH SECTION IN 
MIND AS YOU RATE THE COURSE. 

SECTION 1: 

TO PROVIDE A GENERAL EVALUATION. 

1. THE COURSE AS A WHOLE WAS: 

2. THE COURSE CONTENT WAS: 

3. THE ir^STRUCTOR'S CONTRIBUTION TO THE COURSE WAS: 

4. THE INSTRUCTOR'S EFFECTIVENESS IN TEACHING THE SUBJECT MATTER WAS: 
SECTION 2: 

TO PROVIDE DIAGNOSTIC FEEDBACK TO THE INSTRUCTOR. 

5. COURSE ORGANIZATION WAS; 

6. SEQJENTIAL PRESENTATION OF CONCEPTS WAS: 

7. EXPLANATIONS BY INSTRUCTOR WERE: 

S. INSTRUCTOR'S ABILITY TO PRESENT ALTERNATIVE EXPLANATIONS WHEN NEEDED WAS; 
9. INSTRUCTOR'S USE OF EXAMPLES AND ILLUSTRATIONS WAS: 

10. INSTRUCTOR'S ENHANCEMENT OF STUDENT INTEREST IN THE MATERIAL WAS: 

11. STUDENT CONFIDENCE IN INSTRUCTOr^'S KNOWLEDGE WAS: 

12. INSTRUCTOR'S ENTHUSIASM WAS: 

13. CLARITY OF COURSE OBJECTIVES WAS: 

14. INTEREST LEVEL OF CLASS SESSIONS WAS: 

15. AVAILABILITY OF EXTRA HELP WHEN NEEDED WAS: 

SECTION 3: 

TO PROVIDE INFORMATION ABOUT THE COURSE TO OTHER STUDENTS. 

. 16. USE OF CLASS TIME V^AS: 

17. INSTRUCTOR'S INTEREST IN WHETHER STUDENTS LEARNED WAS: 

18. AMOUNT YOU LEARNED IN THE COURSE WAS: 

19. RELEVANCE AND USEFULNESS OF COURSE CONTENT IS: 

20. EVALUATIVE AND GRADING TECHNIQUES (TESTS. PAPERS. PROJECTS. ETC.) WERE; 

21. REASONABLENESS OF ASSIGNED WORK WAS: 



22, ClARITY of student RESPONSIBILITIES AND REQUIREMENTS WAS: 



SECTION 4: 

OPTIONAL ITEMS--USE ONLY AS DIRECTE:D, 

27. 
28. 
29. 
30. 
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17. 
18. 
19. 
20. 
21. 
22. 



E— EXCELLENT 
VG— VERY GOOD 

G— GOOD 

F— FAIR 

P— POOR 
VP— VERY POOR 
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INSTRUCTOR 



COURSE ... 



EDUCATIONAL ASSESSMENT CENT! 
UNIVERSITY OF WASHINGTON 



SECTION. 



D'RFCTiONS- YOU MAY K^)IJK.\ THIS QUESTIONNAIRE COMPLETELY OR PARTIALLY UNANSWERED WITHOUT PENALTY. 
. ^ Pi^ai AND N:AKE marks FIRiV. but not shiny, do not cross OUT GRIDS. 

00 NOT N:AKr SIRAY MARK^^ ERASE CLEANLY IF YOU CHANGE AN ANSWER. 



WHEN REGiSfERING. WAS THIS A COURSE YOU WANTED TO TAKE? 



YES 



NO 



NEUTRAL 



IS THIS COURSE; 



IN YOUR 
MAJOR 



IN YOUR MINOR OR A DISTRIBUTION AN 

PROGRAM REQUIREMENT REQUIREMEN i ELECTIVE 



OTHER 



SOPHO-'OKE 



JUNiOi^ 



SENIOR 



GRADUATE 



i GRADE YOU EXPECT TO RECEIVE: 



PASS 



IMPORTANT: 



IN RATING THIS COURSE, RESPOND TO EACH ITEM 
CAREFULLY AND THOUGHTFULLY. AVOID LETTING YOl'R 
RESPONSES TO SOME ITEMS INFLUENCE YOUR REFPONSES 
TO OTHERS. KEEP THE PURPOSE OF EACH SPICTIO^J i"' 
MIND AS YOU RATE THE COURSE. 



i SECTION 1- 

} TO PROVIDE A GENERAL EVALUATION. 
; 1. THE COURSE AS A WHOLE WAS: 

: 2. THE COURSE CONTENT WAS: 

; 3. THE INSTRUCTOR'S CONTRIBUTION TO THE COURSE WAS- 

I 4. THE INSTRUCTOR'S Lh ^^ECTIVENESS IN TEACHING THE SUBJECT M.ATTER WAS: 



STUDENT RATING 
FORM 



OTMER 



C 



1. 
2. 
3. 
4. 



E-EXCELLFNT 
VG— VERY GOOD 

G—GOOD 

F— FAIR 

P— POOR 
VP— VERY POOR 



SECTION 2: 

TO PROVIDE DIAGNOSTIC FEEDBACK TO THE INSTRUCTOR. 

5. COURSE ORGANIZATION WAS: 

6. INSTRUCTOR'S PREPARATION FOR CLASS WAS: 

7. INSTRUCTOR AS A DISCUSSION LEADER WAS: 

8. INSTRUCTOR'S CONTRIBUTION TO DISCUSSIONS WAS: 

9. CONDUCIVENESS OF CLASS ATMOSPHERE TO STUDENT LEARNING WAS: 

10. QUALITY OF QUESTIONS OR PROBLEMS RAISED WAS: 

11. STUDENT CONFIDENCE IN INSTRUCTOR'S KNOWLEDGE WAS: 

12. INSTRUCTOR'S EUTHUSIASM WAS: 

13. ENCOURAGEMF.NT GIVEN STUDENTS TO EXPRESS THEMSELVES WAS: 

14. INSTRUCTOrVS OPENNESS TO STUDEN i VIEWS WAS: 

15. INTEREST LEVEL OF CLASS SESSIONS WAS: 

•SECTION 3: 

*^0 PROVIDE INFORMATION ABOUT THE COURSE TO OTHER STUDENTS. 

15. USE OF CLASS TIME WAS: 

17. INSTRUCTOR S INTEREST !N WHETHER STUDENTS LEARNED WAS: 

18. AMOUNT YOU LEARNED IN THE COURSE WAS: 

19. RELEVANCE AND USEFULNESS OF COURSE CONTENT 15: 

20. EVALUATIVE AND GRADING TECHNIQUES {TESTS, PAPERS. PROJECTS. ETC.) WERE: 

21. REASONABLENESS OF ASSIGNED WORK WAS: 

22. CLARITY OF STUDENT RESPONSIBILITIFS AND REQUIREMENTS WAS: 



5. 
6. 
7. 

a. 

9 
10. 

n. 

12. 
13. 
14. 
15. 



16. 
17. 

la. 

19. 
20. 
21. 
22. 



SECTION 4: 

OPTIONAL 


ITEMS--USE ONLY AS DIRECTED. . 

27. 




23. 
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EDUCATIONAL ASSESSMENT CENTER 

UNIVERSITY OF WASHINGTON 

. SECTION 

■ m rHiS QLiESTIONNAIRE COf.iPLETcLY OR HMRriALLY UNANSWERED WITHOUT PENALTY. 
rli-^CIL AND MAK^ MARKS FIRiV BJT NOT SHINY. DO NOT CROSS OUT GRIDS. 
STRAY ."/ARKS ERASE CLEANLY IF YOU CHANGE AN ANSWER. 





r^l"; • THIS A 


COURSE 


YOU WANTED TO 


TAKE? YES NO NEUTRAL 


STUDENT RATING 
FORM 


;5 THIS 


IN YOUR 
• -^-^ MAJOR 


IN YOUR 
PROGRAM 


MINOR OR 
REQUIREMENT 


A DISTRIBUTION AN 
REQUIREMENT ELECTIVE 


OTHER 






SJ^HO 


MORE JUNIO 


SENIOR GRADUATE 


OTHER 


D 




rxPECT TO RECEIVE: 


A 


B C 


D E PASS 
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SECTION J 

TO pr: 

1. tm: 

2. T-^L 

3. T:K 

4. TH-: 



IN RATING THIS COURSE, RESPOND TO EACH ITEM 
CAREFULLY AND THOUGHTFULLY. AVOID LETTING YOUR 
RESPONSES rO SOME ITEMS INFLUENCE YOUR RESPONSES 
TO OTHERS. KEEP THE PURPOSE OF EACH SECTION IN 
MIND AS YOU RATE THE COURSE. 

;/;de a general evaluation. 

.ourse as a ;,hole was: 
Course content was: 

^structor s pontribution to the course was: 

••;':.TRUCTOR'S-.:?'FrECTlVENESS IN TEACHING THE SUBJECT MATTER WAS: 



E—EXCELLENT 
VG— VERY GOOD 

G— GOOD 

F— FAIR 

P— POOR 
VP— VERY POOR 



SECTION 2 
TO PRO 
5. 
6. 
7. 
8. 
9. 



/IDE DIAGNOSTIC FEEDBACK TO THE INSTRUCTOR. 
CO*..= ^::.E ORGANIZATION WAS: 
SE.\ 'ENTIAL PRESEiMATlON OF CONCEPTS WAS: 
EX-'. -'NATIONS BY INSTRUCTOR WERE: 

INt^T.-UCTOR'S ABILITY TO PRESENT ALTERNATIVE EXPLANATIONS WHEN NEEDED WAS; 
INSr-UCTOR'S USE OF EXAMPLES AND ILLUSTRATIONS WAS: 
10. QUALITY OF QUESTIONS OR PROBLEMS RAISED BY INSTRUCTOR WAS: 
IL CONTRIBUTION OF ASSIGNMENTS TO UNDERSTANDING COURSE CONTENT WAS: 

12. INSlRUCTOR'S ENTHUSIASM V7AS: 

13. INSTRUCTOR'S ABILITY TO DEAL WITH STUDENT DIFFICULTIES WAS: 

14. ANSV/ERS TO STUDENT QUESTIONS WERE: 

15. AVAILABILITY OF EXTRA HELP WHEN NEEDED WAS: 

SECTION 3: 

TO PROVIDE INFORMATION ABOUT THE COURSE TO OTHER STUDENTS. 
15, USE OF CLASS TIME WAS. 

17: INSTRUCTOR'S INTEREST IN WHETHER STUDENTS LEARNED WAS: 

18. AVOUNT YOU LEARNED IN THE COURSE WAS: 

19. RELEVANCE AND USEFULNESS OF COURSE CONTENT IS: 

20. EVALUATIVE AND GRADING TECHNIQUES {TESTS, PAPERS, PROJECTS, ETC.) WERE: 

21. REASONABLENESS OF ASSIGNED WORK WAS: 

22. CLARITY OF STUDENT RESPONSIBILITIcS AND REQUIREMENTS WAS: 



1. 

2. 
3. 
4. 



5. 

6. 

7. 

8. 

9 
10. 
11. 
12. 
13. 
14. 
15. 



16. 
17. 
18. 
19. 
20. 
21. 
22. 



SECTION 4: 

OPTIONAL ITEMS-USE ONLY AS DIRECTED. 

27. 
28. 
29, 
30. 
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23. 
24. 
25, 
26. 
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INSTRUCTOR. 



EDUCATIONAL ASSESSMENT CENTER 
UNIVERSITY OF WASHINGTON 



SECTION- 



COURSE - 

DIRECTIONS- YOU MAY RETURN Th.'IS QUESTIONNAIRE COMPLETELY OR PARTIALLY UNANSWERED WITHOUT PENALTY. 
USE A NO 2 PENCIL AND MAKE MARKS FIRM BUT NOT SHINY. DO NOT CROSS OUT GRIDS. 
DO NOT MAKE STRAY MARKS. ERASE CLEANLY IF YOU CHANGE AN ANSWER. 



WHEN REGISTERING, WAS THIS A 


COURSE YOU WANTED TO TAKE? YES NO NEUTRAL 


STUDENT RATING 
FORM 


! ,S THIS COURSE: 


IN YOUR MINOR OR A DISTRIBUTION AN 
PROGRAM REQUIREMENT REQUIREMENT ELECTIVE 


OTHER 


j YOUR CLASS: FRESHMAN 


SOPHOMORE ; JUNIOR SENIOR GRADUAIE 


OTHER 


E 


1 , ^ 

GRADE YOU EXPECT TO RECEIVE: 


; A B . C D E PASS 







IMPORTANT- IN RATING THIS COURSE, RESPOND TO EACH ITEM 

CAREFULLY AND THOUGHTFULLY. AVOID LETTING YOUR 
RESPONSES TO SOME ITEMS INFLUENCE YOUR RESPONSES 
TO OTHERS. KEEP THE PURPOSE OF EACH SECTION IN 
MIND AS YOU RATE THE COURSE. 

SECTION 1: 

TO PROVIDE A GENERAL EVALUATION. 

1. THE COURSE AS A WHOLE WAS: 

2. THE COURSE CONTENT WAS: 

3. THE INSTRUCTOR'S CONTRIBUTION TO THE COURSE WAS: 

.4. THE INSTRUCTOR'S EFFECTIVENESS IN TEACHING THE SUBJ ECT MATTER WAS: 
SECTION 2: 

TO PROVIDE DIAGNOSTIC FEEDBACK TO THE INSTRUCTOR. 

5, OPTORTUNITY FOR PRACTICING WHAT WAS LEARNED WAS: 

6, SEQUENTIAL DEVELOPMENT OF SKILLS WAS: 

7, EXPLANATIONS OF UNDERLYING RATIONALES FOR NEW TECHNIQUES OR SKILLS WERE: 

8, DEMONSTRATIONS OF EXPECTED SKILLS WERE: 

9, INSTRUCTOR'S CONFIDENCE IN STUDENTS' ABILITY WAS: 

10. RECOGNITION OF STUDENT PROGRESS BY INSTRUCTOR WAS: 

11. STUDENT CONFIDENCE IN INSTRUCTOR'S KNOWLEDGE WAS: 

12. FREEDOM ALLOWED STUDENTS TO DEVELOP OWN SKILLS AND IDEAS WAS: 

13. INSTRUCTOR'S ABILITY TO DEAL WITH STUDENT DIFFICULTIES WAS; 

14. TAILORING OF INSTRUCTION TO VARYING STUDENT SKILL LEVELS WAS: 

15. AVAILABILITY OF EXTRA HELP WHEN NEEDED WAS: 

SECTION 3: 

TO PROVIDE INFORMATION ABOUT THE COURSE TO OTHER STUDENTS. 

16. USE OF CLASS TIME WAS: 

17. INSTRUCTOR'S INTEREST IN WHETHER STUDENTS LEARNED WAS: 

18. AMOUNT YOU LEARNED IN THE COURSE WAS; 

19. RELEVANCE AND USEFULNESS OF COURSE CONTENT IS: 

20. EVALUATIVE AND GRADING TECHNIQUES (TESTS, PAPERS, PROJECTS, ETC) WERE: 

21. REASONABLENESS OF ASSIGNED WORK WAS: 

22. CLARITY OF STUDENT RESPONSIBILITIES AND REQUIREMENTS WAS: 



E— EXCELLENT 
VG— VERY GOOD 

G— GOOD 

F— FAIR 

P—POOR 
VP— VERY POOR 



1. 
2. 
3. 
4. 



5. 

6. 

7, 

6. 

9 
10. 
11. 
12. 
13. 
14. 
15- 



16, 
17. 
18. 
19- 
20. 
21. 
22. 



VG 



SECTION 4: 

OPTIONAL ITEMS-USE ONLY AS DIRECTED. 

27. 

28. ; 
29. 

30. 0 
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23. 
24. 
25. 
26. 
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